Chapter Ten 


Control System Theory Overview 


In this book we have presented results mostly for continuous-time, time-invariant, 
deterministic control systems. We have also, to some extent, given the corre¬ 
sponding results for discrete-time, time-invariant, deterministic control systems. 
However, in control theory and its applications several other types of system ap¬ 
peal'. If the coefficients (matrices A, B, C, D) of a linear control system change 
in time, one is faced with time-varying control systems. If a system has some 
parameters or variables of a random nature, such a system is classified as a 
stochastic system. Systems containing variables delayed in time are known as 
systems with time delays. 

In applying control theory results to real-world systems, it is very important 
to minimize both the amount of energy to be spent while controlling a system 
and the difference (error) between the actual and desired system trajectories. 
Sometimes a control action has to be performed as fast as possible, i.e. in 
a minimal time interval. These problems are addressed in modern optimal 
control theory. The most recent approach to optimal control theory emerged 
in the early eighties. This approach is called the II , : optimal control theory, 
and deals simultaneously with the optimization of certain performance criteria 
and minimization of the norm of the system transfer function(s) from undesired 
quantities in the system (disturbances, modeling errors) to the system’s outputs. 

Obtaining mathematical models of real physical systems can be done either 
by applying known physical laws and using the corresponding mathematical 
equations, or through an experimental technique known as system identification. 
In the latter case, a system is subjected to a set of standard known input functions 
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and by measuring the system outputs, under certain conditions, it is possible to 
obtain a mathematical model of the system under consideration. 

In some applications, systems change their structures so that one has first 
to perform on-line estimation of system parameters and then to design a control 
law that will produce the desired characteristics for the system. These systems 
are known as adaptive control systems. Even though the original system may be 
linear, by using the closed-loop adaptive control scheme one is faced, in general, 
with a nonlinear control system problem. 

Nonlinear control systems are described by nonlinear differential equations. 
One way to control such systems is to use the linearization procedure described 
in Section 1.6. In that case one has to know the system nominal trajectories and 
inputs. Furthermore, we have seen that the linearization procedure is valid only 
if deviations from nominal trajectories and inputs are small. In the general case, 
one has to be able to solve nonlinear control system problems. Nonlinear - control 
systems have been a “hot” area of research since the middle of the eighties, since 
when many valuable nonlinear control theory results have been obtained. In 
the late eighties and early nineties, neural networks, which are in fact nonlinear 
systems with many inputs and many outputs, emerged as a universal technological 
tool of the future. However, many questions remain to be answered due to the 
high level of complexity encountered in the study of nonlinear systems. 

In the last section of this chapter, we comment on other important areas 
of control theory such as algebraic methods in control systems, discrete events 
systems, intelligent control, fuzzy control, large scale systems, and so on. 


10.1 Time-Varying Systems 

A time-varying, continuous-time, linear control system in the state space form 
is represented by 


x(f) = A(i)x(f) + B(f)u(f), x(f 0 ) = x 0 

y(t) = C (t)x(t) + D(t)u(t) 


( 10 . 1 ) 


Its coefficient matrices are time functions, which makes these systems much more 
challenging for analytical studies than the corresponding time-invariant ones. 
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It can be shown that the solution of (10.1) is given by (Chen, 1984) 

t 

x(f) = 4>(Mo)x(f 0 ) + J <&(|, t )B(r)u(r )dr (10.2) 

to 

where is the state transition matrix. For an input-free system, the 

transition matrix relates the state of the system at the initial time and the state 
of the system at any given time, that is 

x(i) = <&(/,f 0 )x(f 0 ) (10.3) 

It is easy to establish from (10.3) that the state transition matrix has the following 
properties: 

(1) satisfies the system differential equation 

d 

— $(Vo) = A(f)$(/,/o), $(fo,fo) = I (10.4) 

(2) 4>(Mo) is nonsingular, which follows from 

$ _1 (M 0 ) = $(fo,f) (10.5) 


(3) 4>(Mo) satisfies 


$(Wo) = (10.6) 

Due to the fact that the system matrix, A (t), is a function of time, it is not 
possible, in general, to find the analytical expression for the system state transition 
matrix so that the state response equation (10.2) can be solved only numerically. 

Since the coefficient matrices A(f), B(f), C(f), D(/) are time functions, 
three essential system concepts presented in Chapters 4 and 5—stability, con¬ 
trollability, and observability—have to be redefined for the case of time-varying 
systems. 

The stability of time-varying systems cannot be defined in terms of the 
system eigenvalues as for time-invariant systems. Furthermore, several stability 
definitions have to be introduced for time-varying systems, such as bounded- 
input bounded-output stability, stability of the system’s equilibrium points, global 
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stability, uniform stability, and so on (Chen, 1984). The stability in the sense of 
Lyapunov, applied to the system equilibrium points, x e (f), defined by 

A (Z)Xe(i) = o (10.7) 

indicates that the corresponding equilibrium points are stable if the system state 
transition matrix is bounded, that is 


||*(Mo)|| < const < oo, Vi > t 0 (10.8) 


Since the system transition matrix has no analytical expression, it is hard, in 
general, to test the stability condition (10.8). 

Similarly, the controllability and observability of time-varying systems are 
tested differently to the corresponding ones of time-invariant systems. It is 
necessary to use the notions of controllability and observability Grammians of 
time-varying systems, respectively, defined by 


1 1 

w c (t 0 ,t i) = y 4>(/o--)H(-)H 7 (7-)<!>.(/u. 7 )dr (10.9) 

to 

and 

ti 

^ 1 ) = / ® T (T,to)C T (T)C(T)$(T,t 0 )dT ( 10 . 10 ) 

to 

The controllability and observability tests arc defined in terms of these Grammians 
and the state transition matrix. Since the system state transition matrix is 
not known in its analytical form, we conclude that it is very hard to test the 
controllability and observability of time-varying systems. The reader interested 
in this topic is referred to Chen (1984), Klamka (1991), and Rugh (1993). 

Corresponding results can be presented for discrete-time, time-varying, linear 
systems defined by 


x(& + 1) = A (k)x(k) + B(&)u(&), x(fco) = x 0 
y (k) — C (fc)x(fc) + D(/,')u(A') 

The state transition matrix for the system (10.11) is given by 


( 10 . 11 ) 


$(&, ko) - A(k)A(k —!)••• A (k 0 + 1)A(^ 0 ) 


(10.12) 
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It is interesting to point out that, unlike the continuous-time result, the discrete¬ 
time transition matrix of time-varying systems is in general singular. It is 
nonsingular - if and only if the matrix A(«) is nonsingular' for Vi = kQ,ko+l,...,k. 

Similarly to the stability study of continuous-time, time-varying systems, in 
the discrete-time domain one has to consider several stability definitions. The 
eigenvalues are no longer indicators of system stability. The stability of the 
system equilibrium points can be tested in terms of the bounds imposed on 
the system state transition matrix. The system controllability and observability 
conditions are given in terms of the discrete-time controllability and observability 
Grammians. The interested reader can find more about discrete-time, time- 
invariant and time-varying linear' systems in Ogata (1987). 


10.2 Stochastic Linear Control Systems 

Stochastic linear' control systems can be defined in several frameworks, such as 
jump linear' systems, Markov chains, systems driven by white noise, to name 
a few. From the control theory point of view, linear' control systems driven 
by white noise are the most interesting. Such systems are described by the 
following equations 


x(f) = Ax(t) + Bu(t) + Gw(i), £{x(f 0 )} = x 0 
y(0 = Cx(f) + v(t) 


(10.13) 


where w(f) and v(7) are white noise stochastic processes, which represent the 
system noise (disturbance) and the measurement noise (inaccuracy of sensors or 
their inability to measure the state variables perfectly). White noise stochastic 
processes are mathematical fictions that represent real stochastic processes that 
have a large frequency bandwidth. They are good approximate mathematical 
models for many real physical processes such as wind, white light, thermal noise, 
unevenness of roads, and so on. The spectrum of white noise is constant at all 
frequencies. The corresponding constant is called the white noise intensity. Since 
the spectrum of a signal is the Fourier transform of its covariance, the covariance 
matrices of the system (plant) white noise and measurement white noise are 
given by 


£{w(f)w r (r)} = W 6'fi — r), £{v(t)v r (r)} = V6(t — r) (10.14) 
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It is also assumed that the mean values of these white noise processes are equal 
to zero, that is 

£{w(f)} = 0 , E{v(t)} = 0 (10.15) 

even though this assumption is not crucial. 

The problem of finding the optimal control for (10.13) such that a given 
performance criterion is optimized will be studied in Section 10.3. Here we 
consider only an input-free system, u(t) = 0, subjected to (i.e. corrupted by) 
white noise 


x(f) = Ax(f) + Gw(f), £{x(/ 0 )} = x 0 (10.16) 

and present results for the mean and variance of the state space variables. In 
order to obtain a valid result, i.e. in order that the mean and variance describe 
completely the stochastic nature of the system, we have to assume that white 
noise stochastic processes are Gaussian. It is well known that Gaussian stochastic 
processes are completely described by their mean and variance (Kwakernaak and 
Sivan, 1972). 

Applying the expected value (mean) operator to equation (10.16), we get 

E{k(t)} = A £{x(f)} + E{x(t 0 )} = x 0 (10.17) 

Denoting the expected value £{x(/)j = m(i), and using the fact that the mean 
value of white noise is zero, we get 

m(i)=Am((), m(f 0 ) =-E{x(2o)} = x 0 (10.18) 

which implies that the mean of the state variables of a linear stochastic system 
driven by white noise is represented by a pure deterministic input-free system, 
the solution of which has the known simple form 

in(/) = £{x(/)} = . A <' f = e A ^- 4 °)x 0 (10.19) 

In order to be able to find the expression for the variance of state trajectories 
for the system defined in (10.16), we also need to know a value for the initial 
variance of x(f 0 )- Let us assume that the initial variance is Q(2o) = Qo- The 
variance is defined by 

V<zr{x(t)} = £{[x(i) - m(t)][x(i) - m(2)] r } = Q (t) 


( 10 . 20 ) 
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It is not hard to show (Kwakernaak and Sivan, 1972; Sage and White, 1977) that 
the variance of the state variables of a continuous-time linear system driven by 
white noise satisfies the famous continuous-time matrix Lyapunov equation 

Q CO = Q0)A T + AQ (t) + GWG t , Q(/q) = Q 0 (10.21) 

Note that if the system matrix A is stable, the system reaches the steady state and 
the corresponding state variance is given by the algebraic Lyapunov equation of 
the form 


0 = QA r + AQ + GWG r (10.22) 


which is, in fact, the steady state counterpart to (10.21). 

Example 10.1: MATLAB and its function lyap can be used to solve the 
algebraic Lyapunov equation (10.22). In this example, we find the variance of 
the state variables of the F-8 aircraft, considered in Section 5.7, under wind 
disturbances. The matrix A is given in Section 5.7. Matrices G and W are 
given by Teneketzis and Sandell (1977) 

G = [-46.3 1.214 -1.214 -9.01] T , W = 0.000315 


Note that the wind can be quite accurately modeled as a white noise stochastic 
process with the intensity matrix W (Teneketzis and Sandell, 1977). The 
MATLAB statement 

Q=lyap(A,G*W*G'); 

produces the unique solution (since the matrix A is stable) for (10.22) as 


Q 


0.4731 

-0.0050 

0.0106 

0.0326 


-0.0050 

0.0001 

- 0.0002 

-0.0009 


0.0106 

- 0.0002 

0.0009 

0.0009 


0.0326 ' 
-0.0009 
0.0009 
0.0068 


o 

Similarly, one is able to obtain corresponding results for discrete-time sys¬ 
tems, i.e. the state trajectories of a linear stochastic discrete-time system driven 
by Gaussian white noise 

x(& + 1) = Ax(I’) + Gw (k), £{x(0)} = x 0 , V r ar{x(0)} = Q 0 (10.23) 
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satisfy the following mean and variance equations 

m (k + 1) = Amji'j, m(0) = m 0 => m(&) = A k m 0 (10.24) 

QO + 1) = AQ(k)A T + GWG t , Q(0) = Q 0 (10.25) 

Equation (10.25) is known as the discrete-time matrix difference Lyapunov equa¬ 
tion. If the matrix A is stable, then one is able to define the steady state system 
variance in terms of the solution of the algebraic discrete-time Lyapunov equation. 
This equation is obtained by setting Q {k + 1) = Q (k) = Q in (10.25), that is 

Q = AQA t + GWG t (10.26) 

The MATLAB function dlyap can be used to solve (10.26). The interested 
reader can find more about the continuous- and discrete-time Lyapunov matrix 
equations, and their roles in system stability and control, in Gajic and Qureshi 
(1995). 

10.3 Optimal Linear Control Systems 

In Chapters 8 and 9 of this book we have designed dynamic controllers such that 
the closed-loop systems display the desired transient response and steady state 
characteristics. The design techniques presented in those chapters have sometimes 
been limited to dial and error methods while searching for controllers that meet 
the best given specifications. Lurthermore, we have seen that in some cases it 
has been impossible to satisfy all the desired specifications, due to contradictory 
requirements, and to find the corresponding controller. 

Controller design can also be done through rigorous mathematical opti¬ 
mization techniques. One of these, which originated in the sixties (Kalman, 
1960)—called modern optimal control theory in this book—is a time domain 
technique. During the sixties and seventies, the main contributor to modern opti¬ 
mal control theory was Michael Athans, a professor at the Massachusetts Institute 
of Technology (Athans and Lalb, 1966). Another optimal control theory, known 
as ffoo, is a fiend of the eighties and nineties. H^ optimal control theory stalled 
with the work of Zames (1981). It combines both the time and frequency domain 
optimization techniques to give a unified answer, which is optimal from both the 
time domain and frequency domain points of view (Lrancis, 1987). Similarly to 
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ifoo optimal control theory, the so-called H> optimal control theory optimizes 
systems in both time and frequency domains (Doyle et al., 1989, 1992; Saberi 
et al., 1995) and is the trend of the nineties. Since f/ > optimal control theory 
is mathematically quite involved, in this section we will present results only for 
the modern optimal linear control theory due to Kalman. It is worth mentioning 
that very recently a new philosophy has been introduced for system optimization 
based on linear matrix inequalities (Boyd et al., 1994). 

In the context of the modern optimal linear control theory, we present results 
for the deterministic optimal linear regulator problem, the optimal Kalman filter, 
and the optimal stochastic linear regulator. Only the main results are given 
without derivations. This is done for both continuous- and discrete-time domains 
with emphasis on the infinite time optimization (steady state) and continuous¬ 
time problems. In some places, we also present the corresponding finite time 
optimization results. In addition, several examples are provided to show how to 
use MATLAB to solve the corresponding optimal linear control theory problems. 

10.3.1 Optimal Deterministic Regulator Problem 

In modern optimal control theory of linear deterministic dynamic systems, rep¬ 
resented in continuous-time by 

x(t) = A (f)x(f) + B(f)u(f) s x(f 0 ) = x 0 (10.27) 

we use lineal - state feedback, that is 

u(i) = -F (f)x(t) (10.28) 

and optimize the value for the feedback gain, F (t), such that the following 
performance criterion is minimized 

J = min < - / lx T (7)Rix(7) + u T (7)R?u(f)l dt >, Ri > 0, R? > 0 
u(0 2 J 

to ) 

(10.29) 

This choice for the performance criterion is quite logical. It requires minimization 
of the “square” of input, which means, in general, minimization of the input 
energy required to control a given system, and minimization of the “square” of 
the state variables. Since the state variables—in the case when a linear system is 
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obtained through linearization of a nonlinear system—represent deviations from 
the nominal system trajectories, control engineers are interested in minimizing 
the “square” of this difference, i.e. the “square” of x(/J. In the case when 
the linear mathematical model (10.27) represents the “pure” linear system, the 
minimization of (10.29) can be interpreted as the goal of bringing the system 
as close as possible to the origin (x(f) = 0) while optimizing the energy. This 
regulation to zero can easily be modified (by shifting the origin) to regulate state 
variables to any constant values. 

It is shown in Kalman (1960) that the linear - feedback law (10.28) produces 
the global minimum of the performance criterion (10.29). The solution to this 
optimization problem, obtained by using one of two mathematical techniques 
for dynamic optimization—dynamic programming (Bellman, 1957) and calculus 
of variations—is given in terms of the solution of the famous Riccati equation 
(Bittanti et al., 1991; Lancaster and Rodman, 1995). It can be shown (Kalman, 
1960; Kirk, 1970; Sage and White, 1977) that the required optimal solution for 
the feedback gain is given by 

F opt (t) = -R 2 - 1 B r (/)P(f) (10.30) 

where P(f) is the positive semidefinite solution of the matrix differential Riccati 
equation 

-P(/) = A T (/)P(f) + P{t)A(t) + Ri - P(L)B(/)R 2 'li 7 (/)P(/), P (t f ) = 0 

(10.31) 

In the case of time invariant systems and for an infinite time optimization period, 
i.e. for tf oc, the differential Riccati equation becomes an algebraic one 

0 =A r P + PA + Ri - PBR^B t P (10.32) 

If the original system is both controllable and observable (or only stabilizable and 
detectable) the unique positive definite (semidefinite) solution of (10.32) exists, 
such that the closed-loop system 

x(7) = (A - BR 2 _1 B r P)x(f), x(/ 0 ) = x 0 (10.33) 

is asymptotically stable. In addition, the optimal (minimal) value of the perfor¬ 
mance criterion is given by (Kirk, 1970; Kwakernaak and Sivan, 1972; Sage 
and White, 1977) 


1 
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Example 10.2: Consider the linear deterministic regulator for the F-8 
aircraft whose matrices A and B arc given in Section 5.7. The matrices in 
the performance criterion together with the system initial condition are taken 
from Teneketzis and Sandell (1977) 


Ri = diag{0M 0 3260 3260}, R 2 = 3260 
x 0 = [100 0 0.2 0] T 

The linear - deterministic regulator problem is also known as the linear-quadratic 
optimal control problem since the system is linear - and the performance criterion 
is quadratic. The MATLAB function lqr and the corresponding instruction 

[F,P,ev]=lqr(A,B,Rl,R2); 

produce values for optimal gain F, solution of the algebraic Riccati equation P. 
and closed-loop eigenvalues. These quantities are obtained as 

F = [-0.004 0.5557 -0.2521 0.0590] 


P = 10 4 


0.0000 

-0.0016 

-0.0003 

-0.0003 


-0.0016 

1.6934 

0.1499 

0.2199 


-0.0003 

0.1499 

0.8211 

-0.0713 


-0.0003 

0.2199 

-0.0713 

0.1361 


Ai j2 = -0.9631 ± j'3.0061, A 3 , 4 = -0.0373 ± jO.0837 

The optimal value for the performance criterion can be found from (10.34), 
which produces J op t = 743.9707. 


o 

For linear - discrete-time control systems, a corresponding optimal control 
theory result can be obtained. Let the discrete-time performance criterion for an 
infinite time optimization problem of a time-invariant, discrete-time, linear - system 

x(& + 1) = Ax(fc) + Bu(fc), x(0) = x 0 (10.35) 


be defined by 

X 

J = [x T (fc)R 4 x(&) + u T (fc)R 2 u(&)], R 4 > 0, R 2 > 0 (10.36) 

k =0 
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then the optimal control is given by 

u(k) = - (R 2 + B r PB) _1 B r PAx(A;) = -F opt x(k) (10.37) 

where P satisfies the discrete-time algebraic Riccati equation 

P = A t PA+Ri - A t PB (B t PB + R 2 ) _1 B t PA (10.38) 

It can be shown that the standard controllability-observability conditions imply 
the existence of a unique stabilizing solution of (10.38) such that the closed-loop 
system 

x(k + 1) = (A - BF^)x(fc) (10.39) 

is asymptotically stable. The optimal performance criterion in this case is also 
given by (10.34) (Kwakernaak and Sivan, 1972; Sage and White, 1977; Ogata, 
1987). 


10.3.2 Optimal Kalman Filter 

Consider a stochastic continuous-time system disturbed by white Gaussian noise 
with the corresponding measurements also corrupted by white Gaussian noise, 
that is 


x(2) = Ax(f) + Gw(t), E{x(t 0 )} = x 0 , V a?’{x(f 0 )} = Q 0 
y(t) = Cx(f) + v(f) 


(10.40) 


Since the system is disturbed by white noise, the state space variables arc also 
stochastic quantities (processes). Under the assumption that both the system noise 
and measurement noise arc Gaussian stochastic processes, then so too arc the state 
variables. Thus, the state variables are stochastically completely determined by 
their mean and variance values. 

Since the system measurements arc corrupted by white noise, exact informa¬ 
tion about state variables is not available. The Kalman filtering problem can be 
formulated as follows: find a dynamical system that produces as its output the 
best estimates, x(t), of the state variables x(7). The term “the best estimates” 
means those estimates for which the variance of the estimation error 


e(t) = x(t) — x(2) 


(10.41) 
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is minimized. This problem was originally solved in a paper by Kalman and Bucy 
(1961). However, in the literature it is mostly known simply as the Kalman 
filtering problem. 

The Kalman filter is a stochastic counterpart to the deterministic observer 
considered in Section 5.6. It is a dynamical system built by control engineers 
and driven by the outputs (measurements) of the original system. In addition, it 
has the same order as the original physical system. 

The optimal Kalman filter is given by (Kwakernaak and Sivan, 1972) 

x(7) = Ax(f) + K !;r 40(y(0 - Cx(i)) (10.42) 

where the optimal filter gain satisfies 

K opi (f) = Q(T)C t V -1 (10.43) 

The matrix Q(7) represents the minimal value for the variance of the estimation 
error e(7) = x(/) — x(i), and is given by the solution to the filter differential 
Riccati equation 

Q(t) = AQ (t) + Q(t)A T + GWG T —Q(i)C r V _1 CQ(t), Q (t 0 ) = Q 0 

(10.44) 

Assuming that the filter reaches steady state, the differential Riccati equation 
becomes the algebraic one, that is 

AQ + QA T + GWG r -QC T V“ 1 CQ = 0 (10.45) 

so that the optimal Kalman filter gain K opi as given by (10.43) and (10.45) is 
constant at steady state. 

Note that in the case when an input is present in the state equation, as in 
(10.13), the Kalman filter has to be driven by the same input as the original 
system, that is 


x(7) = Ax(f) + Bu (f) + K ; , pS ( /)(>'(/) - Cx(/)) (10.46) 

The expression for the optimal filter gain stays the same, and is given by (10.43). 

Example 10.3: Consider the F-8 aircraft example. Its matrices A, B, and 
C arc given in Section 5.7, and matrices G and W in Example 10.1. From 
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the paper by Teneketzis and Sandell (1977) we have the value for the intensity 
matrix of the measurement noise 


V = 


0.000686 

0 


0 

40 


The optimal filter gain at steady state can be obtained by using the MATLAB 
function lqe, which stands for the linear-quadratic estimator (filter) design. This 
name is justified by the fact that we consider a linear system and intend to 
minimize the variance (“square”) of the estimation error. Thus, by using 

[K,Q,ev]=lqe(A,G,C,W,V); 

we get steady state values for optimal Kalman filter gain K, minimal (optimal) 
error variance Q, and closed-loop filter eigenvalues. For the F-8 aircraft, these 
are given by 


Ai = 


21.6433 

0.0081 



-0.6020 0.6021 
- 0.0001 0.0001 

0.0053 
- 0.0001 
0.0004 
0.0004 


4.5056 

0.0004 

0.0148 ' 
-0.0004 
0.0004 
0.0031 . 

0.0104 ± j’0.0760 


— 0.0024 
0.0001 
- 0.0001 
-0.0004 

-3.7491, A 2 = -2.6410, A 3j4 


o 

Note that the closed-loop filter in Example 10.3 is asymptotically stable, 
where the closed-loop structure is obtained by rearranging (10.42) or (10.46) as 

x(f) = (A - KepfC)x(f) + Bu(f) + K., p .y(0 (10.47) 

It can be seen from this structure that the optimal closed-loop Kalman filter is 
driven by both the system measurements and control. A block diagram for this 
system-filter configuration is given in Figure 10.1. 

Similarly to the continuous-time Kalman filter, it is possible to develop 
corresponding results for the discrete-time Kalman filter. The interested reader 
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can find the corresponding results in several books (e.g. Kwakernaak and Sivan, 
1972; Ogata, 1987). 



10.1: System-filter configuration 

10.3.3 Optimal Stochastic Regulator Problem 

In the optimal control problem of linear stochastic systems, represented in 
continuous time by (10.13), the following stochastic performance criterion is 
defined 


J = E{ 


1 I 

lim — / [x r (f)Rix(f) + u T (f)R 2 u(f)l dt >, 

tf J 


Ri ^ 0, R 2 ^ 0 
(10.48) 


The goal is to find the optimal value for u(7) such that the performance measure 
(10.48) is minimized along the trajectories of dynamical system (10.13). 


The solution to the above problem is greatly simplified by the application of 
the so-called separation principle. The separation principle says: perform first 
the optimal estimation, i.e. construct the Kalman filter, and then do the optimal 
regulation , i.e. find the optimal regulator as outlined in Subsection 10.3.1 with 
x(/j replaced by x(7). For more details about the separation principle and its 
complete proof see Kwakernaak and Sivan (1972). By the separation principle 




448 


CONTROL SYSTEM THEORY OVERVIEW 


the optimal control for the above problem is given by 

u op t(t) = —F opi x(/) (10.49) 

where F opt is obtained from (10.30) and (10.32), and x(i) is obtained from the 
Kalman filter (10.46). The optimal performance criterion at steady state can be 
obtained by using either one of the following two expressions (Kwakernaak and 
Sivan, 1972) 

J opt = trace {PKVK t + QRi} = trace{PGWG r + QF r R 2 F} (10.50) 

Example 10.4: Consider again the F-8 aircraft example. The matrices F opt 
and P are obtained in Example 10.2 and the matrices K. opt and Q are known 
from Example 10.3. Using any of the formulas given in (10.50), we obtain the 
optimal value for the performance criterion as J opt = 25.0425. The optimal 
control is given by (10.49) with the optimal estimates x(t) obtained from the 
Kalman filter (10.46). 


o 

The solution to the discrete-time optimal stochastic regulator is also obtained 
by using the separation principle, i.e. by combining the results of optimal filtering 
and optimal regulation. For details, the reader is referred to Kwakernaak and 
Sivan (1972) and Ogata (1987). 


10.4 Linear Time-Delay Systems 

The dynamics of linear systems containing time-delays is described by delay- 
differential equations (Driver, 1977). The state space form of a time-delay linear 
control system is given by 

x(f) = A x(t) + Ai)x(t - T) + B u(t) (10.51) 

where T represents the time-delay. This form can be generalized to include 
state variables delayed by 2T, 3 T ,..., time-delay periods. For the puipose of this 
introduction to linear time-delay systems it is sufficient to consider only models 
given by (10.51). 
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Taking the Laplace transform of (10.51) we get 

sX(s) - AX(s) - A D X(s)e“ sT = x(0“) + BU(s) (10.52) 


which produces the characteristic equation for linear time-delay systems in the 
form 


del (.si - A - A^e sT ) = 0 

= s n + d n -i (e sT )s n 1 + • • • + cfi (e sT )s + «o( e ^ s ) = 0 


(10.53) 


Note that the coefficients a,-,i = 0,1,2 ,...,n - 1, are functions of e ~ sT , i.e. 
of the complex frequency s, and therefore the characteristic equation is not 
in the polynomial form as in the case of continuous-time, time-invariant linear 
systems without time-delay. This implies that the transfer function for time-delay 
linear systems is not a rational function. Note that the rational functions can be 
represented by a ratio of two polynomials. 

An important feature of the characteristic equation (10.53) is that it has in 
general, infinitely many solutions. Due to this fact the study of time-delay linear 
systems is much more mathematically involved than the study of linear systems 
without time-delay. 

It is interesting to point out that the stability theory of time-delay systems 
comes to the conclusion that asymptotic stability is guaranteed if and only if all 
roots of the characteristic equations (10.53) are strictly in the left half of the 
complex plane, even though the number of roots may in general be infinite (Mori 
and Kokame, 1989; Su et ah, 1994). In practice, stability of time-delay linear 
time-invariant systems can be examined by using the Nyquist stability test (Kuo, 
1991), as well as by employing Bode diagrams and finding the corresponding 
phase and gain stability margins. 

Studying the controllability of time-delay linear systems is mathematically 
very complex as demonstrated in the corresponding chapter in the book by 
Klamka (1991) on system controllability. Analysis, optimization, and appli¬ 
cations of time-delayed systems are presented in Malek-Zavarei and Jamshidi 
(1987). For control problems associated with these systems the reader is referred 
to Marshall (1977). 

Note that in some cases linear time-delay control systems can be related 
to the sampled data control systems (Ackermann, 1985), which are introduced 
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in Chapter 2, and to the discrete-time linear control systems whose state space 
form is considered in detail in Chapter 3 and some of their properties are studied 
throughout this book. Approximations of the time-delay element e Ts for small 
values of T are considered in Section 9.7. 


10.5 System Identification and Adaptive Control 

System identification is an experimental technique used to determine mathemat¬ 
ical models of dynamical systems. Adaptive control is applied for systems that 
change their mathematical models or some parameters with time. Since system 
identification is included in every adaptive control scheme, in this section we 
present some essential results for both system identification and adaptive control. 


10.5.1 System Identification 

The identification procedure is based on data collected by applying known inputs 
to a system and measuring the corresponding outputs. Using this method, a 
family of pairs (y(U), u(/,-)), i — 1,2,3,... is obtained, where stand for 
the time instants at which the results are recorded (measured). In this section, 
we will present only one identification technique, known as the least-square 
estimation method , which is relevant to this book since it can be used to 
identify the system transfer function of time-invariant linear systems. Many other 
identification techniques applicable either to deterministic or stochastic systems 
are presented in several standard books on identification (see for example Ljung, 
1987; Soderstrom and Stoica, 1989). For simplicity, in this section we study the 
transfer function identification problem of single-input single-output systems in 
the discrete-time domain. 

Consider an nth-order time-invariant, discrete-time, linear system described 
by the corresponding difference equation as presented in Section 3.3.1, that is 


y(k + n) + a n -iy(k + n - 1) +- b a 0 y{k) 

- b n -iu(k + n - 1) + b n - 2 u(k + n - 2) +- b b 0 u(k ) 


(10.54) 


bo] 


It is assumed that parameters 

^ — 1 Un— 2 ' ' ' «0], t) — — 1 b n — 2 


(10.55) 
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are not known and ought to be determined using the least-square identification 
technique. 

Equation (10.54) can be rewritten as 

y (k + n) = + n - 1) -- a 0 y(k) 

+ b n _iu(k + n - 1) + b n _ 2 u{k + n - 2) +-|- b 0 u(k ) 

(10.56) 

and put in the vector form 

y(k + n) = f (k + re. — 1) ^ (10.57) 

where a and b are defined in (10.55), and f( k + n — 1) is given by 


i(k + n — 1) = [— y(k + n — 1} • • • — y(k ) u(k + n — 1) • • • u{k )] 

(10.58) 

Assume that for the given input, we perform N measurements and that the actual 
(measured) system outputs are known, that is 

y a (k + n) 

Y a (&, N) — Vaik + n-l) (10.5 9 ) 

JJa{k + n - N + 1)_ 

The problem now is how to determine 2 n parameters in a and b such that the 
actual system outputs Y a (k,N) are as close as possible to the mathematically 
computed system outputs that are represented by formula (10.57). 

We can easily generate N equations from (10.57) as 

f (k + n — 1) 
i(k + n — 2) 

f (k + n - N + 1) 

i(k t u N) 

Define the estimation (identification) error as 


Y (k,N) = 


y(k + n) 
y(k + n - 1) 


[y(k + n - N + 1). 


= *(^)[b 

( 10 . 60 ) 


E(k, N) = Y a (&, N) - Y(k, N) 


(10.61) 
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The least-square estimation method requires that the choice of the unknown 
parameters a and b minimizes the “square” of the estimation error, that is 

min./ = min j E 7 (A\ A )} (10.62) 

a,b 2 a,b 

Using expressions for the vector derivatives (see Appendix C) and (10.60), we 
can show that 

-V T {k,N)Y a {k,N) = 0 
(10.63) 

which produces the least-square optimal estimates for the unknown parameters as 


min{J} =>• —p—=- = 0 i S T (k,N) i S(k,Ny 



a 

b 


{^ T {k,N)^{k,N)} V(A'.;V)Y a (A'.;Y) 


(10.64) 


Note that the input signal has to be chosen such that the matrix inversion defined 
in (10.64) exists. 

Sometimes it is sufficient to estimate (identify) only some parameters in a 
system or in a problem under consideration in order to obtain a complete insight 
into its dynamical behavior. Very often the identification (estimation) process 
is combined with known physical laws which describe some, but not all, of the 
system variables and parameters. It is interesting to point out that MATLAB 
contains a special toolbox for system identification. 


10.5.2 Adaptive Control 

Adaptive control schemes in closed-loop configurations represent nonlinear con¬ 
trol systems even in those cases when the systems under consideration arc linear. 
Due to this fact, it is not easy to study adaptive control systems analytically. 
However, due to their practical importance, adaptive controllers arc widely used 
nowadays in industry since they produce satisfactory results despite the fact that 
many theoretical questions remain unsolved. 

Two major configurations in adaptive control theory and practice arc self¬ 
tuning regulators and model-reference adaptive schemes. These configurations 
arc represented in Figures 10.2 and 10.3. For self-tuning regulators, it is assumed 
that the system parameters are constant, but unknown. On the other hand, for 
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the model-reference adaptive scheme, it is assumed that the system parameters 
change over time. 



10.2: Self-tuning regulator 



10.3: Model-reference adaptive control scheme 

It can be seen from Figure 10.2 that for self-tuning regulators the "separation 
principle” is used, i.e. the problem is divided into independent estimation and 
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regulation tasks. In the regulation problem, the estimates are used as the true 
values of the unknown parameters. The command signal u c must be chosen 
such that unknown system parameters can be estimated. The stability of the 
closed-loop systems and the convergence of the proposed schemes for self-tuning 
regulators are very challenging and interesting research areas (Wellstead and 
Zanop, 1991). 

In the model-reference adaptive scheme, a desired response is specified by 
using the corresponding mathematical model (see Figure 10.3). The error signal 
generated as a difference between desired and actual outputs is used to adjust 
system parameters that change over time. It is assumed that system parameters 
change much slower than system state variables. 

The adjusted parameters arc used to design a controller, a feedback regulator. 
There are several ways to adjust parameters; one commonly used method is 
known as the MIT rule (Astrom and Wittermark, 1989). As in the case of self¬ 
tuning regulators, model-reference adaptive schemes still have many theoretically 
unresolved stability and convergence questions, even though they do perform 
very well in practice. 

For detailed study of self-tuning regulators, model-reference adaptive sys¬ 
tems, and other adaptive control schemes and techniques applicable to both deter¬ 
ministic and stochastic systems, the reader is referred to Astrom and Wittenmark 
(1989), Wellstead and Zarrop (1991), Isermann et al. (1992), and Krstic et al. 
(1995). 


10.6 Nonlinear Control Systems 

Nonlinear control systems are introduced in this book in Section 1.6, where we 
have presented a method for their linearization. Mathematical models of time- 
invariant nonlinear control systems arc given by 


x(f) = JF(x(f), u(f)), x(f 0 ) = x 0 

y(t) = £(x(f),u(f)) 


(10.65) 


where x(f) is a state vector, uf /) is an input vector, y[t) is an output vector, and 
T and Q arc nonlinear matrix functions. These control systems arc in general 
very difficult to study analytically. Most of the analytical results come from 
the mathematical theory of classic nonlinear differential equations, the theory 
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of which has been developing for more than a hundred years (Coddington and 
Levinson, 1955). In contrast to classic differential equations, due to the presence 
of the input vector in (10.65) one is faced with the even more challenging 
so-called controlled differential equations. The mathematical and engineering 
theories of controlled nonlinear differential equations are the modern trend of the 
eighties and nineties (Sontag, 1990). 

Many interesting phenomena not encountered in linear systems appeal - in 
nonlinear systems, e.g. hysteresis, limit cycles, subharmonic oscillations, finite 
escape time, self-excitation, multiple isolated equilibria, and chaos. For more 
details about these nonlinear phenomena see Siljak (1969) and Khalil (1992). 

It is very hard to give a brief presentation of any result and/or any concept 
of nonlinear control theory since almost all of them take quite complex forms. 
Familiar notions such as system stability and controllability for nonlinear systems 
have to be described by using several definitions (Klamka, 1991; Khalil, 1992). 

One of the most interesting results of nonlinear theory is the so-called 
stability concept in the sense of Lyapunov. This concept deals with the stability 
of system equilibrium points. The equilibrium points of nonlinear systems are 
defined by 

0 - >'(x, (/): u(/ ) j =>• Xg(t) (10.66) 


Roughly speaking, an equilibrium point is stable in the sense of Lyapunov if 
a small perturbation in the system initial condition does not cause the system 
trajectory to leave a bounded neighborhood of the system equilibrium point. The 
Lyapunov stability can be formulated for time invariant nonlinear systems (10.65) 
as follows (Slotine and Li, 1991; Khalil, 1992; Vidyasagar, 1993). 

Theorem 10.1 The equilibrium point x e = 0 of a time invariant nonlinear 
system is stable in the sense of Lyapunov if there exists a continuously differen¬ 
tiable scalar function l(x) such that along the system trajectories the following 
is satisfied 

V(x) > 0, F(0) = 0 


K(x) 


dV_ 

dt 


dV dx 
dx dt ~ 0 


(10.67) 


Thus, the problem of examining system stability in the sense of Lyapunov 
requires finding a scalar function known as the Lyapunov function Lfx). Again, 
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this is a very hard task in general, and one is able to find the Lyapunov function 
V (x) for only a few real physical nonlinear control systems. 

In this book we have presented in Section 1.6 the procedure for linearization 
of nonlinear control systems. Another classic method, known as the describing 
function method, has proved very popular for analyzing nonlinear control systems 
(Slotine and Li, 1991; Khalil, 1992; Vidyasagar, 1993). 


10.7 Comments 

In addition to the classes of control systems extensively studied in this book, and 
those introduced in Chapter 10, many other theoretical and practical control areas 
have emerged during the last thirty years. For example, decentralized control 
(Siljak, 1991), learning systems (Narendra, 1986), algebraic methods for multi- 
variable control systems (Callier and Desoer, 1982; Maciejowski, 1989), robust 
control (Morari and Zafiriou, 1989; Chiang and Safonov, 1992; Grimble, 1994; 
Green and Limebeer, 1995), control of robots (Vukobratovic and Stokic, 1982; 
Spong and Vidyasagar, 1989; Spong et al., 1993), differential games (Isaacs, 
1965; Basai - and Olsder, 1982; Basar and Bernhard, 1991), neural network con¬ 
trol (Gupta and Rao, 1994), variable structure control (Itkis, 1976; Utkin, 1992), 
hierarchical and multilevel systems (Mesarovic et al., 1970), control of systems 
with slow and fast modes (singular perturbations) (Kokotovic and Khalil, 1986; 
Kokotovic et al., 1986; Gajic and Shen, 1993), predictive control (Soeterboek, 
1992), distributed parameter control, large-scale systems (Siljak, 1978; Gajic and 
Shen, 1993), fuzzy control systems (Kandel and Langholz, 1994; Yen et al., 
1995), discrete event systems (Ho, 1991; Ho and Cao, 1991), intelligent vehi¬ 
cles and highway control systems, intelligent control systems (Gupta and Sinha, 
1995; de Silva, 1995), control in manufacturing (Zhou and DiCesare, 1993), con¬ 
trol of flexible structures, power systems control (Anderson and Fouad, 1984), 
control of aircraft (McLean, 1991), linear algebra and numerical analysis con¬ 
trol algorithms (Laub, 1985; Bittanti et al., 1991; Petkov et al., 1991; Bingulac 
and Vanlandingham, 1993; Patel et al., 1994), and computer-controlled systems 
(Astrom and Wittenmark, 1990). 

Finally, it should be emphasized that control theory and its applications are 
studied within all engineering disciplines, and as well as in applied mathematics 
(Kalman et al., 1969; Sontag, 1990) and computer science. 
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